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In  practice  Involves  fewer  underlying  factors  than  eaiHy  In  practice; 

!  Z)  video  games  converge  or  diverge  differentially  upon  one  another;  and 
C  3)  factorial  content  of  the  games  changes  with  practice.  Thus  far,  the 
research  has  resulted  In  two  *  theoretical  paper 
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;7address1ng  the  Issue  of  how  best  to  smooth  correlational  data.  Two  methods 
of  smoothing  Pearson  r's  were  Investigated.  The  first  was  to  average 
repeated  measures  In  blocks  then  correlate  block  averages.  In  the  second 
method  all  repeated  measures  were  correlated;  then  the  correlation 
coefficients  were  averaged  In  blocks.  The  latter  approach  proved  much  superior 
resulting  In  greatly  reduced  sampling  variability,  and  little  distortion  in 
the  population  correlation  estimated. 

The  second  paper  examined  the  question  of  late  appearing  factors 
that  should  occur  after  a  switch  from  controlled  to  automatic  processing.  If 
differential  factors  associated  with  automatic  processing  exist,  they  could 
only  be  observed  after  this  shift  occurs., ^ence,  any  such  factor  would  be 
late-appearing  in  the  sense  that  it  could/ojhly  be  Identified  late  In  practice. 
The  second  paper  reports  two  tests  of  the /existence  of  late-appearing  factors. 
Both  tests  Involved  extended  practice  on  five  video  games;  the  two  tests 
were  carried  out  in  two  different  populations  approximately  one  year  apart. 

The  results  of  the  two  experiments  were  in  complete  agreement.  In  both 
cases  all  factors,  with  one  possible  exception  in  the  second  experiment,  were 
Identified  by  content  exclusively  and  not  by  stage  of  practice.  The  results, 
therefore,  are  negative.  Other  studies/ using  other  materials,  other  subjects, 
or  other  conditions  of  practice  may  re^ch  different  conclusions;  but  the 
studies  reported  In  this  paper  offer  no  support  for  the  existence  of  late- 
appearing  factors. 
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PART  1 


Average  Correlations  Versus  Correlated  Averages 


Abstract 

Vhen  repeated  measures  data  are  collected  with  small  or  even 
moderate  sample  sizes,  correlation  matrices  show  considerable 
variability.  If  one's  primary  interest  is  in  the  correlations,  some 
means  of  smoothing  the  coefficients  may  be  desirable.  Two  methods  of 
smoothing  Pearson  r*s  were  investigated  in  the  present  study.  The  first 
method  was  to  average  repeated  measures  in  blocks  then  correlate  block 
averages.  In  the  second  method  all  repeated  measures  were  correlated 
then  the  correlation  coefficients  were  averaged  in  blocks.  The  latter 
approach  proved  much  superior,  resulting  in  greatly  reduced  sampling 
variability,  and  little  distortion  in  the  population  correlation 
estimated. 


Running  Head :  Average  Correlations 
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In  research  involving  repeated  observations  on  the  same  group  of 
subjects  the  correlations  between  the  various  repeated  measures  are 
often  of  primary  experimental  interest.  Frequently,  however,  one  en¬ 
counters  data  sets  where  there  are  many  repeated  measures  but  rather  few 
subjects.  Given  constraints  on  time  and  expense,  the  number  of  repeated 
measures  and  number  of  subjects  in  practical  research  are  probably 
reciprocally  related.  In  such  cases  the  correlation  matrix  among  re¬ 
peated  measures  may  be  quite  variable,  frustrating  attempts  to  make 
general  statements  about  the  matrix  structure  or  attempts  to  decompose 
that  structure  by  techniques  such  as  factor  analysis.  With  small  to 
moderate  sample  sizes  it  may  be  of  considerable  benefit  to  smooth  the 
correlation  coefficients  prior  to  examining  the  pattern  of 
relationships.  This  poses  the  interesting  question  of  how  best  to 
smooth  these  coefficients,  improving  their  reliability  while  at  the  same 
time  maintaining  their  representation  of  the  underlying  relationships. 

Two  possible  approaches  to  this  problem  were  examined  by  means  of  a 
computer  simulation  followed  by  some  approximate  mathematical  predic¬ 
tions  of  the  statistical  properties  of  the  two  methods.  The  first 
approach  examined  was  to  average  the  data  in  blocks  of  measures  prior  to 
calculating  the  statistic  of  interest.  This  procedure  has  often  been 
used  in  psychological  research  to  smooth  consecutive  observations  and  we 
were  interested  in  how  blocking  of  repeated  measures  would  affect  the 
Pearson  correlation  coefficient.  Blocking  of  data  prior  to  computing 
correlations  is  a  quite  common  procedure  in  many  areas  of  Psychology;  as 
examples  of  this  pratlce  in  the  area  of  motor  learning,  see  Bilodeau 
(1955),  Fleishman  (I960),  or  Reynolds  (1952)-  The  second,  less  obvious, 
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approach  examined  was  to  find  the  simple  correlations  between  all  in¬ 
dividual  repeated  measures,  then  average  these  correlations  in  blocks* 
A  problem  with  the  latter  approach  is  that  the  sampling  distribution  of 
£  is  markedly  skewed  when  the  population  parameter  differs  substantially 
from  0.0,  thus  the  average  £  is  biased  toward  more  moderate  values, 
under  estimating  the  true  population  correlation  (Kendall  &  Stuart, 
1979)*  For  this  reason  the  correlations  were  converted  via  Fisher’s  £ 
transformation,  which  almost  entirely  corrects  skew,  prior  to  averaging. 

METHOD 


The  first  step  in  investigating  the  properties  of  correlations  of 
averages  as  compared  to  average  correlations  involved  a  simulation.  The 
computer  was  programmed  to  generate  12  simulated  "observations"  having  a 
known  common  population  correlation  for  each  of  £  subjects.  Pseudo¬ 
random  normal  deviates  (mean»0,  SD*1 )  were  generated  using  Box  and 
Muller’s  (1958)  procedure.  Seeds  for  the  normal  deviates  were  pseudo¬ 
random  rectangular  fractions  generated  by  the  DEC  System  20  internal 
function  RAN  (see  Edgell,  1979  for  statistical  characteristics).  First, 
12  columns  of  independent  normal  data,  X^j,  were  produced;  then  a  IJth 
column,  was  generated  and  was  added  to  each  of  the  original  12 
columns  to  create  new  intercorrelated  variables,  by  the  following 
formula: 


ij 


It  can  be  shown  that  the  resulting  columns, 


relations  equal  to  r. 


have  population  cor- 


The  12  measures  were  then  averaged  in  blocks  of  2,  3*  4,  or  6,  and 
the  correlations  between  block  means  computed.  Thus,  for  blocks  of 
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three,  the  mean  of  Observations  1,  2,  and  5  was  correlated  with  the 
means  of  Observations  4,  5,  &  6;  7,  8,  A  9;  and  10,  11,  d  12.  The 
program  also  computed  the  correlations  between  all  individual  observa¬ 
tions  without  blocking  (blocks  of  1);  then  these  correlations  were 
averaged  in  blocks  of  2,  3*  4,  or  6.  Thus,  when  the  blocking  factor  was 
3,  the  average  correlation  for  Blocks  1  and  2  was  the  mean  of  9  correla- 
tions,  r, g ,  r^g.  and  r^g.  Whenever 

correlations  were  averaged,  they  were  first  converted  by  Fisher's  ^ 
transformation  to  avoid  distortions  in  the  means  due  to  the  skewed 
sampling  distribution  of 

£  ■  0.5  Ln  [ (1 +£)/( 1 -£)  ] . 

After  averaging,  z  values  were  converted  back  to  £  values  by  the  reverse 
transformation , 

_r  -  [exp(2£)-1 ]/[exp(2£)+1 ]. 

Five  hundred  data  sets  were  generated  for  each  combination  of 
sample  size  (10,  50,  or  80)  and  population  correlation  (0.2,  0.5,  or 
0.6).  For  each  data  set  all  possible  interblock  correlations  were 
computed,  that  is,  66  correlations  for  "blocks'*  of  1  observation,  15  for 
blocks  of  2  observations,  6  for  blocks  of  3  observations,  3  for  blocks 
of  4  observations,  and  1  correlation  for  blocks  of  6  observations.  The 
means  and  standard  deviations  (standard  errors)  of  all  correlations 
resulting  from  a  given  blocking,  over  all  data  sets,  were  then  computed, 
again  using  Fisher's  £.  Ilie  results  are  shown  in  Table  1. 

RESULTS 

As  can  be  seen  in  Table  1,  the  effect  of  averaging  the  data  in 
blocks  prior  to  correlating  produces  an  increase  in  the  block  to  block 
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correlation*  This  increase  in  correlation  as  a  function  of  block  size 
is  entirely  consistent  vith  the  Spearman-Brown  formulSt 

_r(B)  -  kr/[l+(k-l)r], 

where  £(B)  is  the  correlation  of  block  means,  k  is  the  number  of  obser¬ 
vations  per  block,  and  £  is  the  population  correlation  between 
observations.  For  example,  when  the  correlation  equals  0.2,  the  columns 
headed  £(B)  in  Table  1  should  approximately  equal  .20,  .53f  •43»  *50, 
and  .60  respectively  for  blocks  of  size  1,  2,  4,  and  6.  As  can  be 

seen,  when  the  sample  size  is  10,  there  is  a  tendency  to  overestimate 
the  theoretical  values,  but  as  the  sample  size  increases,  the  correla¬ 
tions  between  blocks  closely  approach  their  theoretical  values.  The 
standard  errors  of  the  £(B)'s,  presented  in  the  next  column  of  Table  1, 
depend  only  on  the  sample  size;  and  since  Fisher’s  £  transformation  was 

1  /2 

used,  the  standard  errors  should  approximately  equal  l/(jJ“3)  •  Thus, 

the  standard  errors  should  equal  -378,  .192,  and  .114  for  samples  of  10, 
30,  and  80,  respectively.  Since  £  is  always  calculated  from  the  same 
number  of  data  points,  there  is  no  gain  in  precision  when  the  data  are 
averaged  in  blocks  prior  to  correlating. 


Insert  Table  1  about  here 


The  next  column  of  Table  1,  headed  £,  shows  the  means  of  correla¬ 
tions  averaged  in  blocks.  As  one  would  expect,  these  values  simply 
approximate  the  known  population  correlation,  with  the  exception  that 
with  small  ^  there  is  a  tendency  for  overestimation.  This  small  bias  is 
not  unexpected  since  it  is  known  that  the  mean  of  Fisher  £’ s  is  biased 


r ' 
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in  this  direction  (Kendall  Stuart,  1979)*  Thus,  the  averaged  jr*  a  are 
independent  of  the  blocking  factor,  and  the  bias  toward  overestimation 
is  not  great  even  with  small  sample  sizes*  The  next  column  of  Table  1, 

which  shows  the  standard  errors  of  the  £*s  is  both  the  most  interesting 
and  the  most  complicated.  Clearly,  these  standard  errors  decrease  as  a 
function  of  the  blocking  factor,  thus  there  is  a  gain  in  precision  by 
blocking.  This  gain  in  precision,  however,  is  a  function  of  both  the 
blocking  factor  and  the  population  correlation.  When  the  population 

correlation  is  0.0,  the  standard  error  should  equal  1/[k(N-5)  ]  since 

2 

k  independent  estimates  of  £  are  averaged.  When  the  population  cor¬ 
relation  is  other  than  0.0,  the  estimates  of  £  are  themselves 
intercorrelated ,  which  affects  the  standard  errors  in  such  a  way  that 
less  precision  is  gained  the  larger  the  population  correlation. 

Steiger  (1980)  presents  asymptotic  expressions  for  the  variance- 

2 

covariance  matrix  between  the  k  correlations  to  be  averaged,  however, 
he  did  not  directly  address  the  issue  of  average  correlations  and  their 

2  2 

standard  errors  in  small  samples.  There  are  k  variances,  2k  (k-l) 

2  2 

covariances  between  £*s  sharing  one  common  index,  and  k  (k-l) 
covariances  between  £*  a  with  no  common  indices.  Substituting  the  fact 
that  all  correlations  between  observations  in  the  present  problem  are 

2  2 

equal  into  Steiger's  equations,  one  gets  a  variance  of  (l-£  )  ,  a 

2  3 

covariance  of  £/2(2-£-4£  ‘♦■3£  )  for  correlations  sharing  a  common  index, 

2  2 

and  a  covariance  of  2£  (1-£)  for  correlations  with  no  common  index.  To 
correct  for  the  fact  that  Fisher  z's  were  used,  these  quantities  must  be 
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2  2 

divided  by  )  •  The  variance  of  the  average  £,  V(£)  ,  may  be 

found  by  pre  and  post  multiplying  the  variance-covariance  matrix  by 

2 

vectors  containing  l/k  ;  thus, 

v(r)  -  (1  )r/2(2-r-4r^-^3r^)-^(k-1  )^2r^(  1  -r)^ 

-  "  k2(N-3)(l-?)^"' . 

The  standard  error  is  of  course  the  square  root  of  the  above  expression. 

Using  the  above  formula,  Table  2  was  constructed  which  gives 
theoretical  values  for  the  standard  errors  of  correlations  averaged  in 
blocks  under  a  greater  variety  of  sample  sizes  and  population  correla¬ 
tions  than  were  approximated  in  Table  1.  Further,  Table  2  presents  a 
second  column  headed  EN  which  contains  the  sample  size  required  to 
produce  the  same  accuracy  in  estimating  the  population  correlation  had 
sample  £*s  not  been  averaged;  thus  Column  2  provides  an  estimate  of  the 
“effective  sample  size**  or  efficiency  gained  in  averaging.  These  values 
were  calculated  from  the  following  formula, 

EN  =  1/SE^+5, 

which  is  simply  the  back  solution  from  the  standard  error  of  Fisher’s 
where  ^  is  the  standard  error  and  EN  is  the  effective  sample  size- 

insert  Table  2  about  here 

Comparing  these  theoretical  values  from  Table  2  with  the  empirical 
values  of  Table  1,  particularly  when  the  sample  size  is  smallest,  one 
sees  that  the  theoretical  values  for  the  standard  error  tend  to  be 
slightly  larger,  thus  tend  to  be  somewhat  conservative  in  the  case  of 
small  samples.  It  is  clear  that  substantial  efficiency  is  gained  in 
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averaging  correlations ,  especially  when  the  population  correlation  is 
small  and  the  need  for  power  is  greatest. 

DISCUSSION 

First,  a  consideration  of  the  limitations  of  the  present  study  are 
in  order.  Our  simulation  covers  a  special  case.  Considerable  work 
remains  to  be  done  by  way  of  analyzing  effective  sample  sizes  for  other 
specific  situations,  such  as  correlations  between  two  sets  of  observa¬ 
tions  on  different  measures  that  may  be  differently  correlated  among 
themselves  and  across  measures.  The  major  point  however  i'  lear. 
Average  correlations  have  much  smaller  standard  errors  ( muc  ,^rger 
effective  N*s)  than  correlations  resulting  from  the  commonly  ted 
practice  of  first  averaging  observations. 

The  present  paper  is  concerned  only  with  the  correlations  of  one 
block  of  obsezrvations  with  another  block  of  observations,  thus  in  cal¬ 
culating  r,  k  X  k  correlations  are  averaged.  These  derivations  are  not 
applicable  to  the  average  intercorrelations  within  a  single  block  of 
observations,  where  there  are  k(k-l)/2  correlations  to  average  thus 
different  counting  of  the  various  correlation  types  is  necessary.  This 
latter  problem  is  addressed  by  Bittner  (1962).  Also,  the  present  paper 
does  not  treat  the  issue  of  a  block  of  observations  of  one  task  corre¬ 
lated  with  a  block  of  observations  of  a  second  task.  Here,  some 
correlations  involve  different  observations  of  the  same  task,  whereas 
others  involve  different  tasks;  the  latter  will  tend  to  be  of  a  lower 
magnitude.  Preliminary  investigation  suggests  lower  efficiencies  of  the 
averaging  process  under  these  conditions  (Bittner,  Dunlap,  &  Jones, 
1982). 


Last  it  mav 


that  we  overlooiced  an  obvious  way  to  improve 
some  aspects  of  r(B),  the  coefficient  obtained  by  correlating  means  of 
blocks  of  observations-  The  Spearman-Erown  formula  is  easy  to  reverse; 
thus  from  a  correlation  of  block  means  one  can  estimate  the  unblockec 
correlation  as 

£  =  £l  E) / ^  k-( k-1  ;£(  E)  ^ . 

Preliminary  simulations  were  done  using  this  back  transformation  on 
r(  B) ,  and  two  things  were  discovered.  First,  the  mean  of  the  bacr: 
transformed  r(  B)  *  s  was  badly  biased ,  especially  for  smaller  population 
correlations .  This  bias  could  perhaps  be  corrected  statistically; 
however,  ignoring  the  bias,  the  efficiencies  were  not  nearly  as  great  as 
the  efficiencies  of  averaged  correlations. 

The  conclusions  then  are  rather  clear.  If  one  desires  to  smooth 
correlations,  maintaining  the  representation  of  a  single  observation's 
correlation  with  another,  while  improving  the  coefficient's  accuracy  or 
efficiency,  the  average  correlation  is  superior  to  the  correlation  of 
averages.  It  can  be  argued  that  the  size  of  r(B),  since  it  changes  with 
the  blocking  factor,  is  determined  by  the  experimenter's  fancy;  whereas 

£,  since  it  remains  almost  constant  across  blocking,  is  a  fairer  repre¬ 
sentation  of  the  actual  experimental  unit  usee,  a  single  measure's 
correlation  with  another  single  measure.  Also,  £(  B)  fails  to  satisfy 
another  primary  goal  of  smoothing,  that  of  improving  accuracy  or 
efficiency.  One  simply  arrives  at  a  bigger  correlation  with  the  same 

inherent  instability  using  r(B),  whereas  the  efficiency  of  r  increases 
dramatically,  especially  with  lower  population  correlations  where  gre¬ 
atest  stability  is  really  needed.  Therefore  if  one's  interest  is  in  the 
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pattern  of  correlations  across  observations  or  a  factor  analysis  of 
repeated  measures  data,  and  smoothing  prior  to  analysis  appears  war¬ 
ranted,  averaging  simple  correlations  in  blocks  seems  most  practical. 


11 


REFERENCE  NOTES 

Bittner,  A.  C.,  Jr.  Averaged  correlations  between  parallel  measures: 
Reliability  estimation.  Proceedings  of  the  8th  Psychology  in  LoT 
Symposium,  USA?  Academy,  Colorado  Springs,  CO.,  1982. 

Bittner,  A.  C.,  Jr*,  Dunlap,  ¥.  P. ,  d  Jones,  H.  B.  Averaged  cross¬ 
correlations  with  differentially  stable  variables:  Fewer  subjects 
required  with  repeated  measures.  Proceedings  of  the  26th  annual 
meeting  of  the  Human  Factors  Society,  Seattle,  WA,  1982. 


REyERENCES 


BILODEAU,  E*  A.  Variations  in  knowledge  of  component  performance  and  its 
effects  on  part-part  and  part-whole  relations.  Jo urnal  o f 
Experimental  Psychology,  1955»  215-224. 

BOX,  G.  £•  P. ,  d  MULLER,  M.  E.  A  note  on  the  generation  of  random  normal 
deviates.  Annals  of  Mathematical  Statistics,  1958,  610-611. 

EDGELL,  S.  E.  A  statistical  check  of  the  DEC  System  -  10  FORTRAN  pseu¬ 
dorandom  number  generator.  Behavior  Research  Methods 
Instrumentation ,  1979f  JJ_,  529-550. 

FLEISHMAN,  E.  A.  Abilities  at  different  stages  of  practice  in  rotary 
pursuit  performance.  Journal  of  Experimental  Psychology,  19^0,  60, 
162-171  - 

JONES,  M.  B.  Differential  processes  in  acquisition.  In  E.  A.  Bilodeau 
and  I.  McD. '  Bilodeau  (Eds.),  Principles  of  skill  acquisition.  New 
York:  Academic  Press,  1969- 

KENDALL,  M. ,  &  STUART,  A.  The  advanced  theory  of  statistics,  Vol.  1,  New 
York:  Macmillan,  1979* 

REYNOLDS,  B.  The  effect  of  learning  on  the  predictability  of  psychomotor 
performance.  Journal  of  Experimental  Psychology,  1952,  189-198- 

STEIGER,  J.  H.  Tests  for  comparing  elements  of  a  correlation  matrix. 
Psychological  Bulletin,  1980,  87,  245-251- 


Table  1*  Correlations  Between  Block  Means,  r(B),  and  Correlations 

Averaged  in  Blocks,  r,  with  Respective  Standard  Errors  for 
Population  Correlations  of  0*2,  0.5»  and  0.8,  and  Sample 
Sizes  (N)  Equal  10,  50,  and  80. 


13 


CM 

Q  Ov  O 

Q  A  Q  c*' 
—  ^  00  OD 

QQ 

^  VO  LfN 

o  o 

.  • 

«->  00  vO  vO  vO 

W 

—  O  O 

•  a  « 

^  o  o  o  o 

a  a  •  a  0 

o  o  o  o 

u 

C^  0^  Ov  00 
^  Ov  Cm 

88888 

A  A  A  A  A 

S  S  8  S  8 

o 

^  ^ 

Ifv  lA  LTN  ITN  ifN 

00 

H 

a  #  • 

CM 

a  a 

-r- 

LTV  to  lA  ^ 

A  A  vO  CM 

^  ^ 

^  ^  ^  ^ 

CO 

a  •  a 

a  a 

^  ^  ^  ^ 

OV  00 

00  VO 

QD^ogc^ 

A  ^  A  A  ^ 

CM 

Ch  CT* 

O  vO  A  Q  iA 

Q  0^  CM  ^  vO 
CD  CO  0>  0^  0^ 

u 

to  ^ 

a  «  a 

^  m 

a  a 

A  vo  CO  CO 

u 

A 

8 

A 

VO 

CM 

0^ 

A 

A 

VO 

8 

o 

8 

§ 

VO 

A 

o 

A 

A 

CM 

CO 

a 

O 

a 

o 

O 

a 

a 

QO 

00 

00 

00 

00 

00 

00 

00 

00 

00 

o 

u 

8 

o 

CM 

o 

CM 

O 

CM 

O 

CM 

o 

A 

8 

o 

A 

o 

A 

8 

8 

8 

8 

8 

8 

A 

H 

' 

a 

' 

a 

. 

■ 

‘ 

‘ 

' 

’ 

* 

■ 

S 

EaQ 

A 

CM 

CJv 

CM 

crv 

& 

CM 

CTV 

A 

00 

8 

8 

S 

s 

So 

CO 

a 

a 

- 

A 

O 

A 

VO 

CO 

A 

00 

vO 

CM 

00 

A 

PQ 

o 

A 

A 

Q 

o 

o 

A 

Q 

Q 

CTv 

CM 

VO 

X 

CM 

A 

a 

A 

VO 

A 

VO 

C- 

00 

a 

00 

00 

00 

CJv 

Ov 

Ov 

VO 

A 

00 

00 

A 

CJN 

S 

A 

CJV 

CM 

o 

El] 

r* 

CM 

c- 

A 

CM 

o 

<JV 

VO 

GO 

s 

A 

A 

a 

CM 

a 

a 

A 

CM 

• 

CM 

CM 

A 

CM 

CM 

CM 

CM 

(Jv 

o 

o 

o 

A 

A 

■a* 

O 

O 

CM 

CM 

CM 

CM 

o 

CM 

CM 

CM 

C\J 

a 

CM 

A 

A 

* 

A 

• 

A 

A 

00 

a 

00 

00 

00 

• 

II 

VO 

VO 

00 

CJV 

00 

A 

CM 

A 

A 

<7v 

A 

VO 

CM 

C' 

C- 

C^ 

r* 

c- 

VO 

r- 

VO 

C- 

VO 

CO 

A 

A 

a 

A 

A 

a 

A 

a 

A 

A 

A 

A 

* 

A 

A 

A 

A 

a 

A 

a 

A 

CM 

VO 

VO 

A 

A 

CM 

t- 

CM 

00 

A 

m 

O 

CM 

A 

A 

CM 

A 

CM 

VO 

A 

« 

? 

vO 

VO 

C-- 

s 

a 

CM 

00 

8 

Ch 

a 

CIO 

C 

Pi 

•H 

O 

4» 

^  CM  A  ^  VO 

-i-  CM  A  ^  VO 

CM  A 

O 

O 

O 

03 

iH 

Pu 

A 

a 

C 

o 

o 

•H 

•H 

CM 

A 

00 

-P 

a 

• 

• 

• 

m 

r^ 

o 

o 

o 

fH 

a> 

II 

H 

II 

Pi 

Pi 

Pi 

Pi 

(h 

Table  2.  Standard  Errors  and  Effective  Sample  Sizes  for 
Correlations  Averaged  Over  Blocks  of  Trials* 


o 

H 

U 

CO 


75 

U 


o 

II 

u 


14 


*-  Osl  CM 


CO  GO  LfN  'd'  ^ 
r-  OS  VD  ITN 
CO  CV  CM  CVJ  CM 


to  \o  O  to 
r- 


^  CM  O  O 
0>  lO  CO  to 


vO  lO  VO  CM  CD 

o  o  g 


CM  Q  —  00 

o  30  o 
*-  o  O  O  O 


o  CTk  ^  to 

CM  CM 


CD  lO  CO 

^  CO  CM 
CO  CM  CM  CM  CM 


O  CM  CM 

CM  to  ^  to 


to  vT'  Q  ^  to 
^  m5  US 

CM  *-***-  ^ 


OtOUSlO-r- 
fO  US  <o  CO 


CM  CM  r-  o  to 
OS  CM  CM 


O  O  OS 

US  OS  *-  CM  to 


vO  VD 

O  OS  OS  ® 
*-^-^^0  0  0 


8  cm  VO  CO  CO 
00  CM  US  00 
^  CM  CM  CM 


CM  US  t>  to  O 
O  O  VO  vO 

—  o  o  o  o 


w 

« 

o 

II 

CD 


O  oo  00  CO 
CM  CM  CO 


00  US  ^  oj 
US  ^  ^  00 
to  CM  CM  ^ 


O  O  US  VO 
CM  US  VO  C^ 


to  to  OS  00  r* 
U3  to  CM  -.- 

CM  T-  V-  *- 


O  CM  US  -•*  OS 

to  VO  00  o  ^ 


^  o  O  ^  CO 
OS  to  *-  O  OS 

—  ^  ^  o 


O  to  vO 

US  o  ^  O 


vO  00  o 
^  OS  00 

^  o  a  a  Q 


8r-  00  to  *- 
<0S  US  CM 
*-  CM  CM  to 


CM  00  CD  to 
O  vO  US  US 

—  o  o  o 


O  to  US  vo  to 

O 

CM  00  OS 

O  Cjs  00  O  us 

O  vO  VO 

Q 

CM 

CM 

cc 

CM 

to  ^  vO 

CM 

US  OO  O 

to  CM  to 

US  to  CM  OS  o 

O 

r-  US  o 

d 

^  CM 

*-  CM  CM 

■*“ 

CM  vo 

tl 

Li 

u 

00  US  VD  CM  OS 

to 

CO  to 

'  CM  US  OS  r-  VO 

VO  r-  00  OS  o 

CM 

o  c-  — 

CD 

CM 

US  CM 

-r-  OS  i 

OS  *-  00  vo 

OO  vo  US  US 

o 

vo 

CO  CM 

•  •  « 

CM 

o  o 

•  a  a 

^  ^  o  o  o 

--  o  o  o  o 

O  O  O 

O  vo 

c-  GO  O 

O 

OS  cys  ^  CM 

O  CO 

O  OS  US  O  CTs 

Q 

us  C-  vO 

u 

CM 

VO  ^ 

CM 

US  o  vo  vO 

to  OS  US  *- 

US  US  OS  ^ 

O 

CM  O  Q 

m 

*-  *-  CM 

**  CM  ^ 

^  CM  ^ 

ys  vO  ON 

o 

II 

Li 

cc 

CM 

to  CO  OS  CM 

CM  vO  CO  OS 

vo  o  00  00 

CM 

vo  --  to 

CD 

O 

US  CM  as 

^  to  OS  r-  vO 

OS  o  vo  ^ 

a;  US  ^  to 

O 

US  ?0 

to  CM 

•  • 

^  5 

•  «  • 

C\J 

• 

• 

O  O  O 

•  •  « 

^  ^  o  o  o 

a  •  a  •  a 

^  o  o  o  o 

a 

O  O  O 

a  a  a 

s 

o  ^ 

vO  US  US 

o 

vo  US  US 

O  ^  vo  us  us 

o  *-  vO  US  US 

Q 

vo  us 

O 

u 

to  vO  US 

CM 

US  c-  *- 

to  T-  ^  to 

US  ov  CM  US  OS 

O 

OS  c—  US 

—  CM 

^  CM  vo 

^  CM  ^  (T' 

^  vo 

«-> 

to  ®  us 

d 

Li 

» 

00  OS  VO  ^  to 

to 

o 

CM  vO  ^  00  CM 

vo  to  OS  vo  ^ 

CM 

^  ^  US 

CD 

00 

CM  Os  vo 

CM 

^  sp 

OS  OS  vo  ^  to 

^  ^  to  CM 

O 

US  to  CM 

to  ^ 

^  o  o 

CM 

o  o  o 

^  O  O  O  O 

'1-0000 

o  o  o 

ta 

c 

L4 

•H 

o 

M 

+» 

^  CM 

CO  ^  vo 

CM 

to  ^  vo 

^  CM  CO  ^  vO 

*-  CM  to  ^  VO 

CM  to  ^ 

O 

o 

o 

rH 

Px 

CQ 

(S> 

Q 

rH 

0) 

O 

O 

O 

o 

O 

P< 

CM 

to 

US 

a 

•H 

R 

n 

H 

H 

II 

Si 

CD 

99 

X 

aa 

35 

O 

^  US 

vO 

O 

OS  to  US 

o 

^  O 

r- 

O 

us  us 

Q 

—  to 

^  I 

OS 

35 

CM 

CM  to  to 

to 

to 

^  us  us 

us 

us 

r-  oo 

OS  OS 

o 

us 

o 

L 

» 

00 

vO  vO 

o  S 

vo 

2 

to 

'jC  50 

OS  00  r- 

CM 

CM 

vo  US  *- 

vo 

vO 

CO  o 

to 

CM 

CM 

V 

CM  [ 

CD 

c-- 

C-- 

OS 

US  ^  ^ 

to 

o  o 

O 

GO  r-- 

r- 

r-  ‘ 

CO 

to  CM 

CM 

CM 

CM 

a  a  • 

a 

^ 

a  a  a 

a  a 

o  o 

o 

°  1 

39 

o 

^  VO 

r-  oo 

O 

O  us  00 

O 

^  00 

to 

O 

CJV  CM 

OS  C-- 

O 

OS 

CM 

r-  i 

CO 

» 

*— 

CM 

to  CO  to 

ro 

US  US  vo 

US  OS  OS  o 

o 

US  00 

o 

*—  ! 

7  5495  .026  1480  .055  836  .049 


PART  2 


15 


FACTOKS  APPEARING  lAEE  DJ  PRACTICE 

ABSTRACT 

With  extended  practice  on  a  task  a  shift  seems  to  occur  from  controlled 
to  automatic  processing.  If  differential  factors  associated  with  automatic 
processing  exist,  they  could  only  be  observed  after  this  shift  occurs.  Hence, 
any  sudi  factor  would  be  late-appearing,  in  the  sense  that  it  oould  only  be 
identified  late  in  practice,  “flie  present  paper  reports  two  tests  of  the 
existence  of  late-appearing  factors.  Both  tests  involved  extended  practice  on 
five  video  games;  the  two  tests  were  carried  out  in  two  different  populations 
approximately  one  year  apart.  Ihe  results  of  the  two  experiments  were  in 
complete  agreement.  In  both  cases  all  factors,  with  one  possible  exc^Ttion 
in  the  second  experiment,  were  identified  by  content  exclusively  and  not  by 
stage  of  practice.  Uie  results,  therefore,  are  negative.  Other  studies  using 
other  materieds,  other  subjects,  or  other  conditions  of  practice  may  reach 
different  conclusiois;  but  the  studies  reported  in  this  pe^r  offer  no  support 
for  the  existence  of  late-appearing  factors. 
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In  recent  years  mudi  attention  has  been  focused  on  the  growth  of  auto¬ 
matic  information  processing  with  practice  (Schneider  and  Shiffrin,  1977; 
Dreyfus  &  Dreyfus,  1979;  Jarrett,  1979).  A  task  which  requires  active  oc»itrol 
early  in  practice  may  become  automatic  later  on;  the  task  beoomes  routine  and 
the  skilled  performer  can  attend  to  other  matters.  Individual  differences, 
ho%#ever,  are  found  in  all  b^aviors;  therefore,  some  people  should  become 
better  automatic  processors  than  other  peqple.  If  so  and  if  a  shift  to 
automatic  processing  occurs  late  in  practice,  one  would  expect  some  general 
abilities  to  anerge  late  in  practice  that  could  not  be  identified  earlier  on. 

Factors  eqppearing  late  in  practice  could  have  major  practical  in^rtance. 
It  is  helpful  to  know  how  well  trainees  are  likely  to  do  early  in  training; 
generadly  speaking,  however,  it  is  more  important  to  predict  how  well  they 
will  perform  in  operations,  whoi  training  is  finished.  Factors  associated 
with  automatic  processing  might  well  seirve  the  latter  purpose  and,  if  so, 
would  contribute  considerably  to  the  predictive  validity  of  many  selection 
batteries. 

To  pursue  this  possibility,  the  formal  criteria  for  recognizing  a 
late-aipearing  factor  need  to  be  specified.  Suppose  that  N  subjects  are  given 
extended  practice  on  k  tasks  and  performance  is  scored  at  three  points  (or 
over  three  sets  of  consecutive  trials)  early,  midway,  and  late  in  practice  and 
the  resulting  3k  measures  factor  analyzed.  If  factor  is  late-appearing,  it 
will  correlate  most  strongly  with  different  tasks  late  in  practice  and  less 
strongly  with  those  same  or  other  tasks  earlier  on.  In  all  other  cases  factor 
^  is  not  late-appearing.  If,  for  example,  factor  ^  correlates  strongly 
with  tasks  A  and  B  late  in  practice  and  with  task  C  early  and  midway  in 
practice,  factor  ^  is  not  late-appearing  because  it  can  be  identified  (from 
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task  C)  early  in  practice.  The  only  reservation  is  the  possibility  that  the 
subjects  have  practiced  task  C  or  similar  tasks  and  reached  asynptotic  levels 
before  the  experiment  begins.  In  such  a  case  all  practice  on  task  C  would  be 
late  and  factor  ^  niight  indeed  be  late-^f3pearing.  If,  however,  the  teisks 
under  study  show  conventional  learning  curves  with  practice,  then  factor  2  can 
be  late  appearing  only  if  it  correlates  strongly  with  late  stages  of  practice 
exclusively.  It  need  not  correlate  strongly  with  late  practice  on  all  tasks, 
but  it  must  not  correlate  with  any  teusk  midway  or  early  in  practice,  at  least 
not  as  strongly  as  it  does  with  some  tasks  late  in  practice,  ihe  polar 
opposite  of  a  late-appearing  factor  is  a  factor  which  correlates  most  strongly 
with  one  taek  at  all  stages  of  practice  and  less  strongly  with  other  tasks  at 
any  stage  of  practice.  Such  a  factor  is  identified  by  task-content  exclu¬ 
sively,  without  qualification  as  to  stage  of  practice. 

The  hypothesis  of  late-^>pearing  factors  requires  the  same  subjects  to 
practice  at  least  two  and  preferably  several  tasks.  Only  a  few  studies  in  the 
psychological  literature  meet  this  description  (Duncanson,  1964;  Gundlach, 
1926;  Horn,  1972;  Lewis,  McAllister,  &  Bechtoldt,  1953;  Stake,  1961;  Woodrow, 
1946),  and  none  was  analyzed  with  a  view  to  the  possibility  that  factors  might 
emerge  with  practice  that  could  not  be  identified  earlier  on.  The  present 
study  is  the  first  to  do  so. 

Two  closely  similar  experiments  were  carried  out;  they  will  be  presented 
separately. 

EXPERIMENT  1 

Subjects  and  Procedures 

Eighteen  Navy  enlisted  men  between  the  eK^es  of  19  and  24  and  with  20/20 
corrected  vision  served  as  subjects.  Most  of  the  subjects  had  participated  in 
previous  studies  and  been  exposed  to  critical  tracking,  two^imensional 
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oolflpensatory  tradcing,  and  dual  laboratory  tracking  tasks.  No  subject, 
however,  hed  been  previously  exposed  to  laboratory  pursuit  tracking  tasks  or 
to  microaonputer-television  games.  ^  The  subjects  were  fit  and  notivated. 

They  received  extra  ooifsensation  for  serving  as  subjects  in  a  fully  ef)proved 
research  and  development  program  that  meets  or  exceeds  the  criteria  set  down 
by  the  Navy  concerning  the  protection  of  subjects  in  Secretary  of  the  Navy 
Instruction  3900. 6E  (Thcmas,  Majeski,  Ewing,  and  Gilbert,  1978). 

The  subjects  practiced  five  tasks  one  session  a  day  for  15  consecutive 
working  days.  The  five  tasks  were  all  microcomputer-television  games  manu¬ 
factured  by  Atari,  and  all  were  jaracticed  in  the  same  order  by  all  subjects. 
Air  Combat  Maneuvering  (ACM)  was  practiced  first,  followed  by  Breakout  and 
Surround  taken  concurrently,  followed  by  Race  Car  and  Slalom  also  taken 
ooncurrently.  Table  1  presents  further  detail  on  each  task,  including  trial 
length,  number  of  trials  per  day,  and  score. 

Results 

Means  and  standard  deviations.  Means  and  standard  deviations  for  these  five 
tasks  on  each  day  of  practice  have  alre2dy  been  reported  (Jones,  Kennedy,  & 
Bittner,  1981;  Kennedy,  Bittner,  Harbeson,  &  Jones,  1982).  All  five  show 
sharp  increeises  early  in  practice  followed  by  smaller  and  snaller  increases 
with  continued  preK^ice;  in  short,  all  five  tas)cs  show  conventional  learning 
curves. 

Correlations.  With  reference  to  staiges  of  practice  in  this  experiment,  "early" 
is  defined  as  Days  1-5,  "midway"  as  Days  6-10,  and  "late"  as  Days  11-15. 
Between  any  two  such  stages,  v^ether  of  the  same  task  or  between  two  tasks, 
there  ace  25  correlations.  These  25  correlations  were  averaged  to  obtain  the 
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figures  that  appear  in  Table  2.  Ihe  vjQue  of  .30,  for  exatrple,  which  appears 
in  the  upper  right-hand  comer  of  the  matrix  is  the  averawe  of  the  25  cor¬ 
relations  between  Days  1-5  on  ACM  and  Days  11-15  on  Surround. 

Averaging  was  carried  out  by  first  transforming  eadi  one  of  the  25 
correlations  ty  Fisher's  z,  averaging  the  z-trauisforms,  and  then  transforming 
the  averaige  z  back  to  £.  Ihe  standard  error  of  this  average,  it  should  be 
noted,  is  mudi  less  than  the  value  for  a  single  z-tramsformed  correlation. 
Simulation  study  of  this  question  indicates  that  the  effective  N  for  the 
average  correlation  varies  between  32  and  88  depending  on  the  magnitude  of  the 
correlation  (.90  to  .30}  as  opposed  to  18  regardless  of  magnitude  (Dunlap, 
Jones,  &  Bittner,  submitted). 

The  alternative  to  averaging  oorrelaticxis  would  be  to  average  trials. 

Ihe  sampling  error  associated  with  a  z-transformed  correlation  between  average 
trials  is  the  same  as  for  one  between  single  trieds,  hence  much  larger  than 
the  error  associated  with  an  average  correlaticxi. 

In  addition,  the  average  correlation  estimates  the  average  between-trial 
correlation  regardless  of  how  these  trieds  are  grouped,  whether  in  fives,  as 
in  the  present  case,  threes,  or  what  hawe  you.  Hie  oorrelation  between 
average  trials,  however,  increases  as  the  number  of  triads  being  averaged 
increases  and  is,  therefore,  partly  dependent  on  an  arbitrary  consideration, 
i tamely,  the  way  trials  are  grouped. 

Principal  faK^ors.  To  obtain  the  principal  factors  for  the  correlations  in 
Table  2  oommunalities  were  initiadly  estimated  as  the  squared  multiple 
oorrelation  between  the  variable  in  question  and  the  remaining  14  variables. 
Factoring  was  continued  until  all  remaining  factors  had  eigenvalues  of  less 
than  unity.  Hiis  criterion  yielded  three  factors.  Hie  loadings  on  these 
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three  factors  were  then  used  to  estunate  a  second  set  of  oonmunalities  and  the 
matrix  factored  again  (three  factors).  After  six  iterations  the  oonmunalities 
converged.  The  resulting  factor  loadings  appear  in  Table  3. 

The  three  largest  correlations  for  eadi  factor  in  Table  3  are  marked  with 
an  asterisk.  The  two  laurgest,  .88  amd  .83,  are  for  stages  late  and  midway  in 
practice  on  Breakout.  The  next  largest,  .79,  is  for  late  practice  on  ACM. 

The  next  two,  .76  and. 75,  are  for  early  practice  on  ACM  and  Race  Car.  It  does 
not  seen,  therefore,  that  Factor  1  Ccm  be  regarded  as  late-appearing. 

Factors  2  and  3  are  not  well  identified  at  any  stage  of  practice.  To 
qualify  as  late-appearing  a  factor  must  be  well  identified  late  in  practice 
and  much  less  so  earlier  cm.  None  of  the  principal  factors  meet  this  descrip¬ 
tion,  but  few  psychologists  would  expect  them  to.  If  late-appearing  factors 
exist,  one  would  expect  to  find  them  only  after  rotation  to  some  sort  of 
siirple  structure. 

Rotated  factors.  Table  4  contains  the  correlations  of  each  task  with  the 
varimax  rotated  factors.  The  results  are  unequivocal.  All  three  factors 
exaiplify  vhat  was  described  earlier  as  the  "polar  opposite"  of  a  late 
appearing  factor.  The  three  largest  correlations  are  all  with  the  same  task 
and  all  three  are  considerably  larger  than  the  correlations  with  any  other 
tasks. 

Table  5  contains  the  correlations  of  each  task  with  the  quartimax  rotated 
factors.  Again  the  results  are  unequivoccd. .  Factor  2  is  not  well  identified, 
but  Factors  1  and  3  are  both  clearly  controlled  by  task  content. 

Table  6  contains  the  correlations  of  each  task  with  the  direct  oblimin, 
obliquely  rotetted  factors.  Again,  all  three  factors  are  polar  opposites  of  a 
late-i(]pearing  factor. 
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EXPERIMENT  2 

Subjects  and  Procedures 

Die  subjects  were  63  male  students  taking  introductory  psydiology  clcisses 
at  Tulane  University.  Diey  were  obtained  in  the  following  manner.  All 
students  in  the  class  were  asked  to  ooivplete  a  one-page  questionnaire  re- 
qaurding  previous  experience  with  video  games.  Die  experimenter  then  called 
for  male  volunteers  with  no  more  than  two  hours  of  experience  with  Atari  home 
video  games.  (Die  questionnaires  were  ciiecked  and  volunteers  with  more  thcin 
the  two-hour  limit  were  excused  from  the  experiment.)  Volunteers  were 
informed  that  the  experiment  would  require  two  hours  of  paper-and-pencil 
testing  (not  reported  in  this  study)  and  12  hours  of  video  testing,  that  they 
could  withdraw  at  any  time,  that  taking  part  would  satisfy  the  instructor's 
requirements  for  research  participation,  and  that  volunteers  completed  the 
experiment  would  be  paid  $40  each. 

Video  testing  involved  five  video  tasks:  ACM,  BreaJcout,  Pace  Car, 

Slalom,  and  Antiaircraft.  Die  last  is  game  #1  on  cassette  CX-2602;  each  game 
lasts  2  min  euid  16  sec  and  is  played  with  a  joystick  controller  euid  a  button 
for  firing  at  teurgets.^  Die  five  games  were  located  in  as  many  booths  in  the 
experimented  room,  each  one  connected  to  a  Zenith  19-in  black-and-white 
television  set.  Die  booths  were  separated  at  the  sides  only  by  Celotex 
panels.  Die  subject  sat  1.5  m  from  the  television  set  directly  in  front  of  a 
51x62  cm  table  64  cm  in  height.  Die  Atari  console  and  the  subject's  data 
sheet  also  rested  on  the  table  top. 

Testing  toc^  place  in  12  sessions,  each  one  lasting  approximately  50  min. 

In  each  session  all  five  games  were  played  for  9.5  min  each,  leaving  half  a 
minute  to  change  seats  for  the  next  game.  In  all,  each  subjech  had  12x9.5  > 
114  min  of  ipractice  on  each  game.  Since  each  game  was  played  on  a  separate 
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television  set  in  a  separate  booth,  a  maximum  of  five  subjects  oould  be  tested 
simultaneously.  The  order  of  testing  for  the  five  games  was  determined  by  the 
assignment  of  subjects  to  the  rows  of  a  random  5x5  latin  square  for  each 
experimental  session.  Since  each  letter  (representing  one  of  the  five  games) 
appeared  once  in  each  column  (representing  a  9.5  min  practice  interval),  no 
two  subjects  were  assigned  to  play  the  same  game  at  the  same  time.  A  subject 
was  allowed  no  more  than  two  sessions  on  a  given  day  and  was  required  to 
complete  the  12  sessions  within  six  weeks.  The  order  of  games  for  a  given 
sessicxi  was  written  across  the  top  of  the  data  sheet  used  for  that  session. 

Ihe  games  were  coded  by  the  letters  A  through  E  vAiich  also  appeared  in  large 
blodc  letters  above  the  appropriate  booth.  The  scores  each  subject  achieved 
were  recorded  by  the  subject  in  a  vertical  column  labelled  trials  under  the 
game's  letter.  At  least  cxie  experimenter  was  present  behind  the  subjects  and 
verified  one  or  more  entries  per  player  per  game  in  a  session.  The  subject 
was  responsible  for  starting  a  game  at  the  RESET  signal  and  restarting  the 
game  after  each  recorded  score,  using  the  reset  button  at  the  far  right  of  the 
Atari  oonsole.  The  game  select  button  next  to  the  reset  button  was  blocked  to 
prevent  the  subject  from  inadvertently  shifting  to  another  game.  If  a  subject 
was  in  the  middle  of  a  game  at  the  STOP  signal,  he  recorded  the  incomplete 
score,  then  moved  to  the  next  game  booth. 

For  the  four  timed  games  (ACM,  Race  Car,  Slalom,  and  Antiaircraft)  the 
sJojecr's  scxjre  for  a  session  was  the  average  of  games  played  excluding  the 
first  and  last  game  or  part-game.  For  Breakout,  which  lasts  a  variable  length 
of  time,  the  subject's  scxjre  for  a  session  was  the  total  number  of  points 
acxumulated  in  the  9.5  min. 


Results 
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Means  and  standard  deviations.  Means  and  standard  deviations  for  ACM,  Race 
Car,  and  Slalom,  shown  in  Table  7,  were  similar  to  the  oorresponding  results 
in  Experiment  1.  Since  Breakout  was  scored  by  session  rather  than  by  game  and 
a  session  usually  included  more  than  one  game,  the  scores  on  Breakout  were 
larger  in  Experiment  2  than  in  Experiment  1,  but  the  learning  curves  were  much 
the  same.  Like  the  other  four  games.  Antiaircraft  followed  a  conventional 
learning  curve,  with  the  mean  approximately  doubling  from  the  first  to  the 
twelfth  session. 

Correlations.  In  Experiment  2  the  12  practice  sessions  were  grouped  by  threes 
into  four  stages  of  practice.  Hence,  the  correlations  in  Table  8  are  averages 
of  nine  intersession  correlations.  Averaging  was  done  in  the  same  manner  as 
in  Experiment  1,  that  is,  by  averaging  z-transforms  rather  than  raw  corre¬ 
lations. 

Principal  factors.  Table  9  contains  the  principal-factor  loadings  for  Experi¬ 
ment  2.  The  correlations  v^re  factored  following  the  same  procedures  as  in 
Experiment  1,  this  time  resulting  in  four  factors.  Factors  2,  3,  and  4  are  not 
well  identified,  only  two  of  the  loadings  beina  as  hiah  as  .SO.  The  first 
factor  has  its  highest  four  correlations  with  Stages  3  and  4  on  Race  Car, 

Stage  4  on  Antiaircraft,  and  Stage  1  on  ACM.  The  last  result  excludes  Factor 
1  as  late  appearing.  In  addition,  all  loadings  on  Factor  1  are  relatively 
high  and  distributed  over  a  narrow  range. 

Rotated  factors.  Itie  varimax-rotated  factor  loadings  appear  in  Table  10. 

Three  of  the  four  factors  are  controlled  exclusively  by  content,  the  four 
highest  correlations  all  being  with  the  same  task.  Factor  2  has  its  strongest 
correlation  with  Breakout  and  Race  Car  early  in  practice. 
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The  quartimax  loadings  appear  in  Table  11.  Factors  1,  3,  and  4  show  the 
now-familiar  clustering  of  the  four  highest  correlations  with  the  same  task. 
Factor  2,  however,  meets  the  requirements  for  a  late-appearing  factor.  The 
four  highest  correlations  are  with  Stages  3  and  4  exclusively  and  involve 
three  tasks;  furthermore,  the  correlations  are  substantiad.  Nevertheless,  a 
claim  that  Factor  2  is  late  appearing  seems  unwarranted.  Factor  2  was  not 
late  appearing  in  the  varimax  rotation,  and  in  the  quartimax  rotation  Stage  1 
on  Breakout  has  the  fifth  largest  correlation  with  Factor  2  and  a  close  fifth 
too. 

The  correlations  in  Table  12  with  the  obliquely  rotated  factors  cast 
further  doubt  on  Factor  2  as  late  appearing.  Here  the  largest  correlations 
are  with  Stages  1  and  3  on  Breakout.  Overall,  it  appears  that  Factor  2 
correlates  strongly  with  both  Breakout  and  Race  Car  but,  taking  all  three 
rotations  into  account,  as  strongly  with  Breakout  early  in  practice  as  later 
on.  The  main  difference  between  Factor  2  and  the  other  three  factors  is  that 
the  latter  are  each  defined  by  one  task  (ACM,  Race  Car,  or  Antiaircraft), 
whereas  Factor  2  appears  to  be  defined  by  two  tasks  (Brea3«jut  and  Race  Car). 


DISCUSSION 

All  of  the  tasks  studied  in  this  report  demand  less  attention  once  they 
stabilize  than  they  do  earlier  on.  Stabilization  is  achieved  when  the 
individual  learning  curves  become  parallel  except  for  random  perturbations  and 
the  mean  learning  c:urve  is  increasing  slowly  if  at  all  (Jones,  Kennedy,  and 
Bittner,  1981).  When  this  point  is  reached,  a  subject's  performance  is  no 
longer  changing  appreciably  either  absolutely  or  relative  to  the  performance 
of  other  individuals.  In  one  of  the  first  studies  to  use  video  games  for 
psychological  experimentation  Rebert  and  Low  (1978)  remarked  that  "Excellent 
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play  also  appears  to  involve  the  autoniatization  of  perfornrance,  as  300-400 
hits  can  be  adiieved  during  simultaneous  conversation."  Rebert  and  Low  were 
commenting  on  Pong,  not  one  of  our  six  games  but  one  we  have  studied  elsewhere 
(Note  1).  Our  experience  is  the  same  eis  Rebert  and  Low's,  not  only  for  Pong 
but  also  for  the  games  studied  here.  All  of  these  tasks  stabilize  with  much 
less  practice  than  the  amounts  given  in  our  two  studies  (Note  1;  Jones, 
Kennedy,  and  Bittner,  1981;  Kennedy,  Bittner,  Harbeson,  and  Jones,  1982). 

Once  they  do  stabilize,  moreover,  a  reserve  for  carrying  out  other  functions 
not  only  becomes  availcdsle  but  may  present  problems  for  some  uses  of  these 
tasks  (Note  2).  The  beisis  of  automatization,  unfortunately,  is  not  as  clear 
as  the  fact  itself.  One  cannot  say,  for  example,  that  automatization  depends 
on  consistent  mapping,  as  one  can  about  the  tasks  studied  by  Sciineider  and 
Shiffrin  (1977).  Tlie  idea  of  consistent  mapping  does  not  apply  to  the  tasks 
studied  in  this  report  or,  if  it  does,  can  be  brouaht  to  bear  only  after 
prolonged,  detailed,  and  extremely  difficult  analysis. 

The  present  studies  are  not  conclusive,  of  course.  All  of  the  tasks 
studied  in  this  report  can  be  learned  in  a  relatively  short  time  and  all  are 
similar  in  psydrological  requirements  as  well  as  the  equipment  used.  One 
cannot  be  sure  that  if  different  tasks  were  studied,  different  results  might 
not  be  obtained.  Nevertheless,  as  far  as  they  go,  the  results  are  clear;  no 
new  factors  emerge  with  practice.  As  it  stands,  this  conclusion  is  liable  to 
be  confused  with  other,  similar  sounding  but  definitely  different,  proposi¬ 
tions.  Three  such  propositions  need  to  be  discussed  specifically. 

First,  an  absence  of  ] ate-a^:pearing  factors  does  not  mean  that  no  factor 
grcws  stronger  with  practice.  Fleishman  arri  Rich  (1963),  for  example,  admin¬ 
istered  a  spatial  test  and  a  measure  of  kinaesthetic  sensitivity  to  40  college 
students,  v*k)  were  then  given  40  1-min  trials  on  the  Two-Hand  Coordination 


Test  (raC).  Tile  measure  of  kinaesthetic  sensitivity  correlated  more  and  more 
strongly  with  TOC  as  practice  progressed,  beginning  with  a  value  of  .03  for 
the  first  four  triads  and  ending  with  a  value  of  .40  for  the  last  four  trials. 
Kinaesthetic  sensitivity  clearly  relates  more  strongly  to  TOC  late  than  early 
in  practice.  It  is  not,  however,  a  new  factor.  The  hypothesis  under  test  is 
that  factors  appeau:  with  practice  that  could  not  be  identified  earlier  an. 

In  the  Fleishman  and  Rich  study  kinaesthetic  sensitivity  was  measured  before 
practice  began.  Conceivably,  a  test  that  either  involved  or  presupposed 
extensive  practice  might  measure  a  late-appearing  factor.  To  prove  the  point, 
however,  one  would  have  to  show  that  it  did,  in  fact,  emerge  with  practice  in 
the  sense  that  we  are  using  the  word  "energe."  One  would  have  to  show  that 
the  factor  loaded  heavily  only  after  extensive  practice  on  a  specified  set  of 
tasks. 

The  second  proposition  concerns  "specific"  variance.  If  two  tasks  are 
both  practiced,  the  correlation  between  them  late  in  practice  represents 
"ocmmon"  variance;  all  other  variance  in  the  two  taslcs  is  specific.  If  more 
than  two  tasks  are  practiced,  as  in  our  case,  the  oommunality  for  trials  late 
in  practice  represents  oommon  variance,  and  what  remains  is  specific.  Except 
for  extreme  Ccises,  the  hypothesis  under  test  carries  no  implications  for 
specific  variance.  Late-appearing  factors  involve  common  variance.  Hence, 
the  appearance  of  factors  late  in  practice  is  incon^tible  with  tot^d  speci¬ 
ficity.  To  the  extent,  however,  that  any  common  variance  exists  late  in 
practice,  so  may  late-appearing  factors.  It  is  generally  agreed,  for  example, 
that  gross  motor  perfortianoe  after  extended  practice  is  largely  task  specific 
(Henry  &  Nelson,  1956;  Lindeburg,  1949).  Nevertheless,  between-task  cor¬ 
relations  late  in  practice  frequently  range  as  high  as  .40  or. 50,  certainly 
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high  enough  to  accxxmodate  late-appearing  factors  if  any  exist.  Of  courser 
the  absence  of  late-appearing  factors  (the  conclusion  we  have  reached)  is 
fully  conpatible  with  high,  even  total  task  specificity. 

The  third  proposition  is  more  general ,  namely,  that  the  absence  of 
late-appearing  factors  does  not  exclude  differential  change  with  practice;  and 
in  our  two  studies  both  within-  and  between-task  correlations  definitely  do 
change  with  practice.  In  Experiment  2,  for  example,  all  five  tasks  show 
"superdiagonal  form"  (Jones,  1969);  that  is,  the  smallest  correlation  is  in 
the  upper  right-hand  comer  of  the  matrix  (between  the  first  and  fourth  stages 
of  practice),  vAiile  the  largest  correlations  are  in  the  superdiagcxial  (between 
neighboring  stages).  In  both  experiments  Breakout  correlates  itore  strongly 
with  ACM  late  than  early  in  practice;  that  is.  Breakout  "converges"  on  ACM 
(Jones,  Kennedy,  &  Bittner,  Note  3) .  In  Experiment  2  Race  Car  and  Anti- 
aircTeift  converge  on  each  other.  The  question  then  arises  as  to  how  these  two 
things  can  be  reconciled.  How  is  it  possible  for  the  correlations  between  and 
within  tasks  to  change  if  no  new  factors  emerge  with  practice? 

There  are  three  major  answers  to  this  question.  The  first  is  that  error 
ocirponents  in  the  practiced  teisks  may  change  with  practice.  Suppose,  for 
example,  that  they  become  smaller;  all  reliable  components  remain  the  same 
but  error  variance  shrinks.  In  such  a  case  all  correlations  between  tasks 
will  increase  because  all  correlations  will  become  less  attenuated  with 
practice.  Simileurly,  of  course,  an  increase  in  error  variance  (a  reduction  in 
reliability)  would  mean  smaller  correlations  late  than  early  in  practice. 

The  second  possibility  is  that  different  abilities  (different  factors) 
are  t^ped  as  practice  proceeds.  This  is  the  usual  interpretation  of  changing 
correlations  with  practice,  the  one  adopted  by  Fleishmem  and  Rich  in  their 
study  of  kinaesthetic  sensitivity  and  by  Fleishman  and  other  co-workers  in 
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numerous  other  studies  (for  example,  Fleishman,  1960;  Fleishman  &  Henpel, 
1954;  Fleishman  &  Parker,  1959).  In  all  these  studies  the  abilities  at  issue 
are  identified  and  mecisured  before  practice  begins.  Subjects  utilize  dif¬ 
ferent  abilities  at  different  stages  of  practice,  hence,  task  correlations 
change,  but  nothing  new  emerges.  A  task  singly  taps  a  different  mix  of 
abilities  as  practice  progresses. 

The  third  possibility  is  that  the  same  abilities  are  involved  at  dif¬ 
ferent  stciges  of  practice  but  the  subjects  change  in  relation  to  them.  In 
factor-analytic  terms  it  is  the  factor  scores  that  change,  not  the  factors 
thanselves.  As  practice  proceeds,  subjects  move  to  new  positions  on  some 
factors  (abilities);  in  varying  degrees  they  improve  and,  as  they  do,  the 
correlations  between  taslcs  change.  Theoretically  this  third  possibility  is 
quite  distinct  from  the  second;  it  seems  competent  to  the  facts  and  has  its 
atdherents  (Alvares  and  Hulin,  1972,  1973;  Dunham,  1974).  Unfortunately,  as 
these  authors  have  themselves  pointed  out,  a  general  empirical  test  that  would 
discriminate  between  the  changing-tausk  (Fleishman)  and  changing-subject  models 
is  difficult  to  inagine;  in  any  case,  no  one  has. 

What  matters  for  the  present  paper  is  that  the  existence  or  nonexistence 
of  late-appearing  factors  is  a  different  question  than  eary  previously  raised 
in  this  field.  Changing  correlations  with  practice  can  be  and  have  been 
explained  in  several  ways  without  resort  to  late-appearing  factors.  If 
late-appearing  factors  could  be  demonstrated,  it  would  certainly  2u:gue  in 
favor  of  the  changing-task  conception,  though  in  a  sense  never  intended  by  ary 
of  its  proponents  to  date.  If,  on  the  other  hand,  late-e^ipearing  factors  do 
not  exist,  the  fact  in  no  way  gainsays  the  changing  task  model;  it  does  not 
gainsay  the  changing-subject  model  either.  The  existence  or  nonexistence  of 
late-appearing  factors  is  an  indeperxSent  question. 
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Finally,  two  technical  points  need  to  be  made.  In  all  our  tasks  a  single 
global  measure  of  performance  was  used.  If  late-*appearing  factors  exist,  one 
v'Hild  like  to  measure  them  directly.  That  would  meem  using  other,  more 
specific  measures.  As  long,  however,  as  the  question  at  issue  is  the  exist¬ 
ence  of  late-appearing  factors  a  battery  of  psychologically  more  unitary 
measures  is  premature,  the  central  claim  of  factor  analysis  is  that  it  ceui 
detect  latent  (unmeasured)  variations;  and  the  design  we  have  used  allows  us 
to  conclude  v*iether  or  not  late-appearing  latent  variations  exist.  If  the 
findings  had  favored  the  existence  of  such  variations,  the  next  step  would 
have  been  to  try  to  measure  them  directly.  As  it  is,  however,  any  such 
attempt  is  pointless. 

In  the  factor  analyses  we  carried  out  factoring  was  continued  until  all 
remaining  factors  had  eigenvalues  of  less  than  unity.  This  is  the  conven¬ 
tional  procedure.  Nevertheless,  it  leaves  open  the  possibility  that  some 
factors  with  small  eigenvalues  might  load  most  heavily  in  late  practice  only. 
Ihey  could  hardly,  however,  load  heavily  in  an  absolute  sense  at  any  point. 
Even  if  sudi  factors  existed,  they  oould  not  be  identified  in  the  usual  sense 
of  the  word.  One  oould  not,  for  example,  calculate  factor  scores  that  would 
serve,  even  roughly,  as  a  surrogate  for  the  factor  itself. 

From  a  factor-analytic  point  of  view  the  results  of  the  present  study  are 
encouraging.  If  practice  brought  with  it  new  factors,  those  factors  oould  not 
be  measured  by  the  usued  brief  tests.  In  order  to  get  at  them  one  would  have 
to  provide  extended  practice,  an  expensive  and  laborious  process.  A  failure, 
therefore,  to  find  late-appearing  factors  is  oonsonant  with  existing  proce¬ 
dures  . 
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Prcm  a  training  point  of  view,  the  same  results  are  not  so  enoouraging. 
The  army  is  currently  experimenting  with  video  games  as  part-teisk  trainers 
(Trachtman,  1981).  But  if  practice  does  not  produce  new  factors,  what  is  the 
basis  for  treuisfer  of  training  to  operational  tasks?  In  the  absence  of  new 
factors,  the  only  possibility  is  the  third  interpretation  of  differential 
change  with  practice  mentioned  earlier.  That  is,  if  transfer  of  training 
takes  place,  it  must  be  that  subjects  are  improving  their  scores  along  lines 
(abilities,  factors)  that  could  have  been  identified  before  practice  began. 
Presunably  their  scores  at  that  point  would  be  lower.  It  may  be,  too,  that 
these  abilities  can  only  be  measured  through  video-oomputer  as  contrasted  with 
more  conventional  tasks.  In  this  sense,  the  current  wave  of  video-coirputer 
tasks  may  uncover  new  dimensions  of  human  skills  and  abilities;  but  practicing 
the  tasks  seems  not  to  produce  any  further  novelties. 
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FOOTOOTES 

^Data  cxjllection  for  this  study  began  in  the  f2dl  of  1980  v*jen  fully 
progrannable  video  syst«ns  for  heme  television  sets  had  just  arrived 
on  the  market  and  were  still  relatively  unfamiliar. 

^Both  left  euid  right  difficulty  switches  were  set  in  the  A  position  for 
edl  five  games  in  Experiment  2. 
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TABLE  1 


Correlations  among  five  microcomputer-television  tasks,  early,  midway,  and  late  in 
practice.  Experiment  1. 


No< 

.  Task 

stage  of 
Practice  1 

2  3  4  5  6  7  8  9  10  11  12  13  14  15 

1 

ACM 

Early 

.80  .82  .48  .64  .66  .49  .41  .40  .46  .26  .12  .21  .38  .30 

2 

Midway 

-  .91  .46  .57  .57  .41  .32  .31  .36  .09  .05  .22  .41  .29 

3 

Late 

-  .55  .64  .65  .44  .43  .40  .34  .17  .10  .21  .46  .35 

4 

Breakout 

Early 

-  .78  .73  .43  .40  .37  .23  .43  .22  .00  .43  .20 

5 

Midway 

-  .82  .59  .50  .48  .43  .44  .21  .08  .43  ,33 

6 

Late 

-  .61  .58  .54  .50  .56  .32  .13  ,50  .37 

7 

Race  Car 

Early 

-  .65  .65  .53  .46  .40  .20  .46  ,45 

8 

Midway 

-  .72  .37  .48  .43  .31  .45  .47 

9 

Late 

-  .35  .42  .44  .24  .42  .49 

10 

Slalom 

Early 

-  .43  .28  .19  .27  .25 

11 

Midway 

-  .57  .18  .32  .10 

12 

Late 

-  .31  .39  .16 

13 

Surround 

Early 

-  .31  .21 

14 

Midway 

-  .44 

15 


Late 
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TABLE  3 


Principal 

factors  for  the  correlations 

in  Table  2. 

Factor 

Task 

Practice 

I - 

2 

ACM 

Early 

.76 

-.39 

.08 

Midway 

.71 

-.57* 

.20 

Late 

.79* 

-.51* 

.16 

Breakout 

Early 

.69 

0.11 

-.45* 

Midway 

.83* 

-.14 

-.34* 

Late 

.88* 

-.03 

-.28 

Race  Car 

Early 

.75 

.24 

.08 

Midway 

.72 

.34 

.18 

Late 

.69 

.33 

.18 

Slalom 

Early 

.54 

.10 

.01 

Midway 

.55 

.46 

-.28 

Late 

.42 

.52* 

.03 

Surround 

Early 

.28 

.16 

.33* 

Midway 

.60 

.10 

.14 

Late 

.48 

.09 

.30 

♦The  three  largest  correlations  for  each  factor  are  marked  with  an  asterisk. 


39 


TABLE  4 


Varinax-rotated  factors  for  the  correlations  in  Table  2. 


Task 

Stage  of 

Practice 

1 

Factor 

- 2 - 

— r“ 

ACM 

Early 

.76* 

.29 

.25 

Midway 

.91* 

.12 

.18 

Late 

.90* 

.22 

.23 

Breakout 

Early 

.40 

.72* 

.06 

Midway 

.52 

.72* 

.18 

Late 

.48 

.73* 

.31 

Race  Car 

Early 

.28 

.42 

.61* 

Midway 

.20 

.35 

.71* 

Late 

.19 

.33 

.68* 

Slalom 

Early 

.24 

.33 

.37 

Midway 

-.09 

.64 

.41 

Late 

-.15 

.33 

.56 

Surround 

Early 

.12 

-.06 

.45 

Midway 

.31 

.26 

.48 

Late 


28 


06 


50 


40 


TABLE  5 

Quart iniax-rotated  factors  for  the  oorrelations  in  Table  2. 


Task 

Stage  of 

Practice 

T - 

Factor 

- J 

ACM 

Early 

.62* 

.12 

.57 

Midway 

.80* 

.03 

.48 

Late 

.76* 

.08 

.57 

Breakout 

Early 

.23 

.58* 

.65 

Midway 

.32 

.51* 

.68 

Late 

.24 

.44* 

.77* 

Race  Car 

Early 

.04 

.02 

.79* 

Midway 

-.04 

-.09 

.81* 

Late 

-.04 

-.10 

.77* 

Slalom 

Early 

.08 

.07 

.54 

Midway 

-.31 

.30 

.63 

Late 

-.33 

-.03 

.57 

Surround 

Early 

.02 

-.30 

.36 

Midway 

.13 

o 

• 

1 

.61 

Late 

.13 

-.22 

.52 

*The  three  largest  oorrelations  for  each  factor  are  marked  with  an  asterisk. 

Pour  correlations  with  Factor  3  are  marked  because  of  the  tie  for  third  largest. 


TABLE  6 


Direct  oblimin  (oblique)  factor  structure  for  the  oorrelations  in  Table  2. 


Task 

Steige  of 

Practice 

T - 

Factor 

- 2 - 

- r 

ACM 

Early 

.79* 

.48 

.46 

Midway 

.92* 

.34 

.38 

Late 

.92* 

.44 

.45 

Breakout 

Early 

.42 

.79* 

.31 

Midway 

.56 

.83* 

.45 

Late 

.52 

.86* 

.57 

Race  Car 

Early 

.33 

.57 

.74* 

Midway 

.26 

.51 

.80* 

Late 

.25 

.47 

.77* 

Slalom 

Early 

.28 

.43 

.48 

Midway 

-.04 

.67 

.53 

Late 

-.10 

.38 

.58 

Surround 

Early 

.14 

.04 

.43 

Midway 

.35 

.40 

.58 

Late 

.31 

.21 

.55 

*The  three  largest  oorrelations 

for  each  factor  are 

marked  with 

an  asterisk. 

Trial  means  and  standard  deviations  of  five  microoonDuter-television  tasks.  Experiment  2 
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Task 


Stage  of 
Practice 


Factor 


ACM 

1 

.70* 

-.01 

-.25 

.03 

2 

.66 

.22 

-.50* 

-.04 

3 

.65 

.26 

-.44* 

-.04 

4 

.64 

.25 

-.36* 

-.03 

Breakout 

1 

.66 

-.14 

.04 

.31* 

2 

.67 

.03 

-.01 

.29 

3 

.69 

-.18 

-.10 

.27 

4 

.69 

-.05 

-.09 

.19 

Race  Car 

1 

.54 

-.24 

.00 

.22 

2 

.60 

-.34* 

.21 

.14 

3 

.72* 

-.25 

.24 

.03 

4 

.77* 

-.24 

.15 

-.09 

Sledom 

1 

.58 

.23 

.20 

.09 

2 

.61 

.40* 

.36* 

.07 

3 

.53 

.58* 

.31 

-.04 

4 

.50 

.49* 

.25 

-.09 

Antiaircraft 

1 

.54 

-.30 

.06 

-.25 

2 

.68 

-.12 

.10 

-.30* 

3 

.68 

-.23 

-.04 

-.38* 

4 

.75* 

-.11 

.02 

-.32* 

*1110  four  largest  correlations 

for  each  factor 

are  marked  with 

an  eisterisk 

! 
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TABLE  10 


Varimax-rotated  factors  for  the  correlations  in  Table  8. 


Stage  of 

Factor 

Task 

Practice 

1 

2 

3 

4 

ACM 

1 

.52* 

.40 

.15 

.31 

2 

.79* 

.20 

.17 

.19 

3 

.75* 

.18 

.22 

.19 

4 

.69* 

.19 

.26 

.19 

Breakout 

1 

.22 

.65* 

.19 

.20 

2 

.32 

.58* 

.29 

.13 

3 

.33 

.64* 

.10 

.24 

4 

.38 

.53 

.20 

.24 

Race  Car 

1 

.16 

.56 

.05 

.23 

2 

-.01 

.60* 

.11 

.40 

3 

.06 

.56 

.25 

.51 

4 

.17 

.50 

.23 

.60 

Sledom 

1 

.18 

.32 

.52* 

.17 

2 

.13 

.28 

.74* 

.16 

3 

.20 

.08 

.81* 

.11 

4 

.20 

.05 

.70* 

.16 

Antiaircraft 

1 

.12 

.26 

.05 

.61* 

2 

.21 

.24 

.26 

.64* 

3 

.29 

.21 

.10 

.72* 

4 

.32 

.25 

.24 

.67* 

♦The  four  largest  correlations  for  each  factor  are  marlced  with  an  asterisk. 
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TABLE  11 


Quart imax-rotated  factors  for  the  oorrelations  in  Table  8. 


Stage  of 

Factor 

Task 

Practice 

1 

i  3 

4 

ACM 

1 

.34* 

.66 

.04 

.00 

2 

.66* 

.54 

.10 

.01 

3 

.63* 

.52 

.15 

.02 

4 

.56* 

.52 

.18 

.01 

Breakout 

1 

-.01 

.70 

.04 

-.23 

2 

.11 

.66 

.16 

-.25 

3 

.10 

.74* 

-.05 

-.19 

4 

.16 

.68 

.06 

-.13 

Race  Car 

1 

-.04 

.61 

-.08 

-.14 

2 

-.24 

.68 

-.04 

-.03 

3 

-.19 

.77* 

.09 

.08 

4 

-.08 

.80* 

.07 

.19 

Slcdom 

1 

.03 

.50 

.43* 

-.06 

2 

-.01 

.49 

.66* 

-.06 

3 

.11 

.34 

.77* 

.01 

4 

.12 

.33 

.66* 

.07 

Antiaircraft 

1 

-.06 

.58 

-.06 

.34* 

2 

.02 

.65 

.13 

.37* 

3 

.10 

.66 

-.02 

.46* 

4 

.12 

.71* 

.13 

.38* 

♦The  four  largest  oorrelations  for  eacii  factor  are  marked  with  an  asterisk 


47 


TABLE  12 


Direct  oblimin  (oblique)  factor  structure  for  the  correlations  in  Table  8. 


Stage  of 

Task  Practice 

1 

Factor 
“1 - 

1 - 

4 

ACM  1 

.63* 

.57 

.37 

.53 

2 

.85* 

.42 

.39 

.43 

3 

.82* 

.40 

.43 

.42 

4 

.76* 

.41 

.45 

.42 

Breakout  1 

.37 

.73* 

.38 

.46 

2 

.47 

.68 

.47 

.41 

3 

.46 

.74* 

.32 

.50 

4 

.51 

.66 

.40 

.49 

Race  Car  1 

.27 

.62 

.22 

.43 

2 

.15 

.68 

.28 

.57 

3 

.25 

.71* 

.43 

.69 

4 

.37 

.69* 

.44 

.78* 

Sl5dcni  1 

.34 

.46 

.62* 

.38 

2 

.33 

.44 

.81* 

.37 

3 

.37 

.26 

.84* 

.29 

4 

.36 

.24 

.74* 

.31 

Antiaircraft  1 

.24 

.44 

.22 

.67 

2 

.37 

.48 

.43 

.74* 

3 

.43 

.46 

.31 

.80* 

4 

.48 

.52 

.46 

.80* 

*11ie  four  largest  correlations 

for  eadi  factor 

are  marked 

with 

an  asterisk, 

Summary  and  Conclusions 
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Widespread  availability  of  high  speed,  large  storage,  economic  computers 
has  set  the  stage  for  dramatic  change  in  the  assessment  of  differential  abilities. 
It  appears  clear  that  assessment  procedures  controlled,  administered,  and  scored 
by  computer  will  shortly  become  a  viable  alternative  to  paper-and-pencil  tests. 
Computer  testing  opens  also  the  prospect  of  assessing  capacities  not  measurable 
with  paper-and-pencil;  in  particular,  computer  testing  allows  the  assessment  of 
motor  skills  on  tasks  modeled  after  video  games.  The  present  research  involved 
administering  extended  practice  on  five  commercial  video  games,  as  well  as  a 
battery  of  paper-and-pencil  tests.  The  research  questions  were;  1)  Does  the 
factorial  content  of  the  video  games  change  with  practice  and,  if  so,  is  there  a 
late  appearing  factor(s)  that  ties  in  with  an  hypothesized  shift  from  controlled 
to  automatic  processing  with  practice  ?  2)  Do  video  games  involve  new  common 
factors  which  have  not  been  and  perhaps  cannot  be  identified  in  paper-and-pencil 
tests  ?  3)  To  what  extent  can  video-game  performance  be  predicted  from  conven¬ 
tional  paper-and-pencil  tests  ? 

Findings  -  Part  1 .  Two  methods  of  smoothing  Pearson  r's  were  investigated  by 
a  Monte  Carlo  computer  simulation.  One  method  was  to  average  repeated  measures 
in  blocks  and  then  to  correlate  block  averages.  In  the  other  method  all  repeated 
measures  were  correlated  and  then  the  correlation  coefficients  were  averaged  in 
blocks,  after  converting  via  Fisher's  z_  transformation  to  avoid  bias.  The  latter 
approach  proved  much  superior,  resulting  in  greatly  reduced  sampling  variability, 
and  little  distortion  in  the  estimated  population  correlation.  Therefore,  if  one's 
purpose  is  to  describe  the  relation  between  learning  trials,  averaging  correlations 
produces  a  much  better  result  than  correlating  averages. 

Findings  -  Part  2.  Part  2  of  this  report  examined  the  possibility  of  late 
appearing  factors  consequent  to  a  switch  from  controlled  to  automatic  processing. 

If  differential  factors  associated  with  automatic  processing  develop,  they  could 
exist  only  after  this  switch  and,  hence,  could  be  identified  only  late  in 
practice.  Two  tests  of  this  reasoning  were  carried  out;  both  used  extended  prac¬ 
tice  on  five  video  games,  but  in  two  different  populations  and  approximately  one 
year  apart.  The  results  of  the  experiments  are  in  striking  agreement.  In  both 
cases,  with  one  possible  exception  in  the  second  experiment,  factors  were  identi¬ 
fied  by  content  exclusively  and  not  by  stage  of  practice.  Other  studies  using 
other  materials,  subjects,  or  conditions  of  practice  may  reach  different  conclu¬ 
sions;  however,  the  studies  reported  in  this  paper  offer  no  support  for  the 
existence  of  late-appearing  factors  and  may  be  interpreted  to  challenge  the  notion 
of  an  identifiable  shift  from  controlled  to  automatic  processing. 


We  are  currently  evaluating  the  relations  of  the  one-time  paper-and-pencil 
tests  to  the  various  stages  of  practice  on  the  video  games,  with  the  aim  of 
identifying  and  characterizing  that  portion  of  motor  skills  performance  that  is 
not  predictable  from  the  one-time  tests.  This  would  be  the  kind  of  information 
that  skills  tasks  might  add  to  a  standard  test  battery. 

Suggestions  for  Future  Research.  We  envision  three  major  directions  for 
pursuing  these  studies.  First,  more  information  is  needed  on  the  relations 
between  paper-and-pencil  tests  and  video  games.  Our  work  represents  only  a 
beginning,  as  we  used  one  battery  of  ":onventional  tests  out  of  a  large  number  of 
possible  test  batteries.  We  also  used  only  one  set  of  video  games  and  there  are 
many  others.  Second,  retention  and  learning  studies  are  needed  to  determine 
the  temporal  stability  of  video- computer  skills.  How  well  are  the  test  retained 
on  the  average,  and  how  well  does  performance  at  the  end  of  acquisition  correlate 
with  performance  at  retention  ?  Both  questions  must  be  asked  for  varying  lapses 
of  time  between  acquisition  and  retention  testing.  Third,  the  games  themselves 
need  to  be  extended.  There  is  no  way  to  modify  existing  commercial  video  games  to 
suit  experimental  purposes.  Similarly,  the  scoring  of  a  player's  performance  still 
must  be  recorded  by  hand.  Close  facsimilies  to  many  of  the  commercial  games  are 
available  on  the  Apple  II  or  other  personal  computers,  and  a  burgeoning  market 
now  exists  for  game  programs  written  specifically  for  personal  computers.  These 
new  developments  should  be  exploited  in  the  interests  of  greater  experimental 
control  and  flexibility  of  scoring. 
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